Papers with information richness
HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing paradigm to fine-tune parameters of pre-trained language models poses problems in data-scarce and resource-limited scenarios. |
| Approach: | They propose a parameter-efficient fine-tuning method HiFi that fine-tails only the highly informative and strongly correlated attention heads for the specific task. |
| Outcome: | The proposed method obtains state-of-the-art over the prior benchmarks on the GLUE benchmark. |
Towards Comprehensive Description Generation from Factual Attribute-value Tables (P19-1)
Copied to clipboard
| Challenge: | Existing models for comprehensive descriptions for factual attribute-value tables might suffer from missing key attributes and groundless information problems. |
| Approach: | They propose a force attention method to encourage the generator to pay more attention to uncovered attributes to avoid potential key attributes missing. |
| Outcome: | The proposed model outperforms the state-of-the-art baselines on automatic and human evaluation. |